Why Self-Driving Cars Must Be Programmed to Kill

"davesaddiction @ opposite-lock.com" (davesaddiction) 10/26/2015 at 12:10 • Filed to: automotivatons

1

49

Self-driving cars are already cruising the streets. But before they can become widespread, carmakers must solve an impossible ethical dilemma of algorithmic morality.

MIT Technology Review

!!! UNKNOWN CONTENT TYPE !!!

450X_FTW > davesaddiction @ opposite-lock.com 10/26/2015 at 12:16

4

Problem solved

Justin Hughes > davesaddiction @ opposite-lock.com 10/26/2015 at 12:19

2

I think of this as kind of like Asimov’s Zeroth Law of Robotics:

A robot may not harm humanity, nor, through inaction, allow humanity to come to harm.

It adds the moral compass of harming or allowing harm to come to a human being, in violation of the First Law, if the end result will benefit humanity as a whole. (Would a robot thusly programmed go back in time and kill Hitler?)

In the case of autonomous cars, this would be more like the theme from Star Trek II : “The needs of the many outweigh the needs of the few, or the one.”

Smallbear wants a modern Syclone, local Maple Leafs spammer > davesaddiction @ opposite-lock.com 10/26/2015 at 12:22

3

They should be programmed to ignore jaywalkers. Hit them.

While this may seem like a cold-hearted view:

- It would bring home the fact that BY STEPPING INTO A ROAD WHERE YOU HAVE NO BUSINESS BEING YOU ARE PUTTING YOURSELF IN DANGER.

- When there is no driver to be blamed jaywalkers would have to live with their mistakes on their own means. People would probably stop jaywalking. Or at least, the blatent fuck-you-traffic type of jaywalking, which is the kind I have problems with.

- It would make self-driving cars behaviour more predictable.

Or it could be jaywalkers have pissed me off one too many times, I dunno.

Daily Drives a Dragon - One Last Lap > davesaddiction @ opposite-lock.com 10/26/2015 at 12:30

3

That’s really similar to the one about the train car on the track.

davesaddiction @ opposite-lock.com > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 12:33

1

I work in downtown Tulsa. Traffic is fairly light, except at peak times, and jaywalking is a regular occurrence and not enforced. Do I do it? Yeah, sometimes, if it’s clearly clear to cross (lots of one way streets). Are some people complete idiots about it? Yes.

Some history: http://www.bbc.com/news/magazine-…

davesaddiction @ opposite-lock.com > Daily Drives a Dragon - One Last Lap 10/26/2015 at 12:34

0

yup

jariten1781 > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 12:35

0

That’s how it typically works now. Drivers are almost never prosecuted for hitting pedestrians unless it was at a clearly marked intersection during daylight with witnesses. Drivers say they didn’t seen them and if they’re some place without a crosswalk/end of a street the police lean towards the driver’s POV. Juries are a crapshoot if the driver is sued, but they lean towards the driver as well if the pedestrian was in a non-crossing zone.

Hasn't stopped jay walkers.

davesaddiction @ opposite-lock.com > Justin Hughes 10/26/2015 at 12:36

0

Do you take action to save three people if it will kill one in the process?

What is the one is your mother/sister/daughter...?

These are questions computers can’t answer, and society as a whole really has no answer for either.

Daily Drives a Dragon - One Last Lap > davesaddiction @ opposite-lock.com 10/26/2015 at 12:36

3

It’s going to be morally sketchy because at some point, someone has told the car to either kill you or kill someone else.

SidewaysOnDirt still misses Bowie > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 12:39

0

...and cyclists.

PS9 > davesaddiction @ opposite-lock.com 10/26/2015 at 12:44

0

I predict morans are going to try to get SDCs to hit them in a bid for free insurance monies.

Smallbear wants a modern Syclone, local Maple Leafs spammer > jariten1781 10/26/2015 at 12:45

0

Here, I believe you are always at fault because there are no clear laws regarding crossing points. Iirc the law just says something about a “crossing point withing reasonable distance”. I guess you could fight it and win, but it would be hard.

If they were run down all the time with no consequences, they would stop. Or, natural selection would take its course.

TylerLinner > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 12:45

1

http://www.vox.com/2015/1/15/7551…

I agree with you about some people who choose to jump out in front of traffic, however it’s helpful to be cognizant of the history of this issue before recommending hard-wired manslaughter.

Smallbear wants a modern Syclone, local Maple Leafs spammer > SidewaysOnDirt still misses Bowie 10/26/2015 at 12:48

0

Cyclists I don't usually mind. They move faster, and are actually supposed to be on the roads anyway. Vespa's, on the other hand... too slow for the road too fast for the sidewalk as smug and unpredictable as a prius driver

Funktheduck > davesaddiction @ opposite-lock.com 10/26/2015 at 12:50

1

Let the lawsuits begin!

This will get interesting

SidewaysOnDirt still misses Bowie > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 12:50

1

Depends. Cyclists are supposed to follow the road rules and at least here, I’ve had so many of them them jump from the sidewalk into oncoming traffic head-on waving wildly at all of the cars moving at 50 mph as though they’re all supposed to just stop on a dime for him. If cycling laws were enforced like traffic laws are, I might not mind as much.

Teleph0nes > PS9 10/26/2015 at 12:53

0

Well, morons currently try to get normal cars to hit them for insurance money. Given how much data a true autonomous car would be constantly monitoring, it would probably be able to show the offending pedestrian’s path as they timed trying to jump in front of the car. Basically it’s covered in dash cams that are also lasers and radar and such.

Smallbear wants a modern Syclone, local Maple Leafs spammer > TylerLinner 10/26/2015 at 12:54

0

Regardless of how it came about, the jaywalking laws make sense.

The machine you are walking in front of weighs 10 or more times what you do and travels many times faster than you. For the mutual safety of everybody, stay out of the roads!!! Basically what I'm saying is if there were real, undefendable and permanent consequences to jaywalking, the issue should sort itself out. In any case re-read the last line of my last comment... this is semi serious/semi blow-off-steam rant.

Smallbear wants a modern Syclone, local Maple Leafs spammer > SidewaysOnDirt still misses Bowie 10/26/2015 at 12:55

0

For those guys, hit.

davesaddiction @ opposite-lock.com > Daily Drives a Dragon - One Last Lap 10/26/2015 at 13:00

0

Pretty much.

Sampsonite24-Earth's Least Likeliest Hero > davesaddiction @ opposite-lock.com 10/26/2015 at 13:00

0

Just program them using Asimov’s 3 laws of robotics. What could possibly go wrong

Roman Savchuk > davesaddiction @ opposite-lock.com 10/26/2015 at 13:01

1

That’s simple. With autonomous stuff getting smarter and people getting dumber it’s a matter of time when cars will be better at making decisions, and if something goes wrong they could respond with “Who are you to judge me, silly stupid human being?”

davesaddiction @ opposite-lock.com > Sampsonite24-Earth's Least Likeliest Hero 10/26/2015 at 13:04

1

Nothing, clearly.

davesaddiction @ opposite-lock.com > Roman Savchuk 10/26/2015 at 13:05

0

Logical perfection...

jariten1781 > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 13:12

1

You in the US? Everywhere I’ve lived in the US has the same boilerplate language to the effect of (taken from VA):
The driver of any vehicle on a highway shall yield the right-of-way to any pedestrian crossing such highway:

1. At any clearly marked crosswalk, whether at mid-block or at the end of any block;

2. At any regular pedestrian crossing included in the prolongation of the lateral boundary lines of the adjacent sidewalk at the end of a block;

3. At any intersection when the driver is approaching on a highway or street where the legal maximum speed does not exceed 35 miles per hour.

No pedestrian shall enter or cross an intersection in disregard of approaching traffic.

There's also language about having to follow traffic lights.

It’s very driver friendly even though the common wisdom and what is taught in drivers ed is ‘Pedestrians always have right of way’. That’s demonstrably false, just no one reads the law. Police/courts tend to rule in line with the law.

TylerLinner > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 13:12

1

Saying people should stay out of the road, and saying drivers (whether they be human or AI) should aim for them are two very different things. It frightens me that ideas like yours are floating out there, serious or not.

As a cycle commuter, I know that all too many people are ignorant of, or outright disregard, their obligation to yield to peds and bicycles. I know that they are too preoccupied on their phones to bother looking where they’re going. So, to think that an autonomous car might want to kill me, well, I take that as a threat.

Let’s start by building infrastructure that works for ALL road users, so people don’t feel the need to jaywalk. Does it make sense to have to walk a half mile to the nearest crosswalk, as people sometimes have to around my area? Does it make sense for sidewalks to simply dead end into fields, chain link fences, ravines or active construction sites (I’ve seen all of these)? These are the issues we should be going after before we sic the cops on people.

Look, I know you and most others on this site like to joke around and “blow off steam”, but as someone who uses this infrastructure daily, I would like to start a meaningful dialogue on this. Instead of calling for consequences, we should be asking why people act the way we do. Is it 100% because they don’t care? I know there is a fraction of road users, in cars or not, that disregards nearly all laws. But there is also a network of underlying problems that combine to make “Proper” road use workable at best, and impossible at worst.

Smallbear wants a modern Syclone, local Maple Leafs spammer > jariten1781 10/26/2015 at 13:14

1

Canuckistan. Ontario to be specific.

Dr. Zoidberg - RIP Oppo > davesaddiction @ opposite-lock.com 10/26/2015 at 13:17

1

Solution? Save the manuals.

Storz > davesaddiction @ opposite-lock.com 10/26/2015 at 13:22

0

I have absolutely no desire ever for a self driving car. Ever.

davesaddiction @ opposite-lock.com > Dr. Zoidberg - RIP Oppo 10/26/2015 at 13:27

0

Word.

gin-san - shitpost specialist > davesaddiction @ opposite-lock.com 10/26/2015 at 13:33

1

I never thought driving was so difficult or stressful that autonomous driving is necessary; if anything, I would treat it as a fancy cruise control. It’s like they discussed on Top Gear - planes can do most things on their own, but I would never want to take a flight without a pilot. I’m much more stressed as a passenger than I am as a driver (I guess it’s a “control” thing) and the only time I really want autonomous driving is when I’m in bumper to bumper traffic - it’d be nice to just sit back and monitor the car’s behavior rather than do the constant coast/brake/coast/brake shuffle.

Smallbear wants a modern Syclone, local Maple Leafs spammer > TylerLinner 10/26/2015 at 13:40

0

I’m not saying they should aim for jaywalkers.I’m saying in these hypothetical situations where the car has a choice of crashing or running down jaywalkers, the car should hit the jaywalkers.

Re the bikes, you would have an advantage here as the car does know the laws. In any case, they would be safer, as:

- A bike is moving faster than a jaywalker, and thus the closing speed is less, giving the car more time to react.

- A bike tends to be moving WITH rather than ACROSS traffic, which again makes it easier for the car to account for.

And agreed. And imo, tunnels/bridges make far more sense than crosswalks. Unfortunately these are also far more expensive... Does it make sense to have the nearest crosswalk 1/2 a mile away? Well, if there are stores on the other side you need to get to, perhaps not. If there isn’t anything there, and you just need the other side because you need the next turn, why not go to the intersection? On the store point, I don’t think it would apply in most situations, at least here. If there are stores there are crosswalks/intersections with crosswalks near enough.

As far as discussion goes, let me start by putting my idea out without hyperboly:

- If the car can possibly avoid the pedestrian, it will do it.

- If the pedestrian cannot be avoided, they will be run down. (Think about it another way. If a pedestrian walks out in front of a car which will force it to kill the occupants, does that not sound remarkably like murder to you? As opposed to suicide, which is what running them down would be like. In any case, I firmly believe that the onus is on the pedestrian to make sure they are behaving in a safe manner, not on the car to “fix” their mistakes.)

- In situations where a crash would not be fatal (or likely to cause serious injury to the occupants) the crash would be the prefered option to avoid taking a life.

- At all areas where the sidewalk ends on one side, that area should be designated as a crosswalk to allow safe passage to continue a journey on the other side of the road.

- For all areas where the sidewalk doesn’t continue (eg. it only exists to allow access to an alleyway into a residential area) that should be marked clearly after the manner of “No Exit” roads.

Debate.

davesaddiction @ opposite-lock.com > gin-san - shitpost specialist 10/26/2015 at 13:48

1

I enjoy driving, but like you said, slow-speed autopilot for traffic jams would be great. Also, and I’m not sure I’d ever trust it enough to do this, the thought of sleeping while my car drives through the night, cross-country on the interstate, does have some appeal to me. Time is valuable.

Daily Drives a Dragon - One Last Lap > davesaddiction @ opposite-lock.com 10/26/2015 at 13:48

0

Autonomous cars raise big moral questions.

davesaddiction @ opposite-lock.com > Daily Drives a Dragon - One Last Lap 10/26/2015 at 14:02

0

Which hopefully will keep them from being the only option for a long, long time. But related, is it moral to drive your own car if it’s been proven than a computer-driven car is 100 times less likely to get into a fatal accident...? I don’t want to answer that question.

davesaddiction @ opposite-lock.com > Storz 10/26/2015 at 14:03

0

I’m not sure I’d ever trust it enough to do this, but the thought of sleeping while my car drives through the night, cross-country on the interstate, does have some appeal to me. Time is valuable.

Daily Drives a Dragon - One Last Lap > davesaddiction @ opposite-lock.com 10/26/2015 at 14:04

0

We should just skip the autonomous car part and make good public transit everywhere. And bike share.

davesaddiction @ opposite-lock.com > Daily Drives a Dragon - One Last Lap 10/26/2015 at 14:06

0

I agree, to a point. Good public transportation only makes sense where there are enough people to use it to make it worthwhile. There are few to no people in many of the places I like to visit.

swaptastic > davesaddiction @ opposite-lock.com 10/26/2015 at 14:17

0

The car will always have to protect the occupants at all costs. One reason being this “should” always be the goal of auto manufacturing from start; to transport people from one place to another as safe as possible. The other reason is because if we develop cars to always take in account the number of lives saved, this can be used against the driver. If someone knew the occupant was driving alone they and an accomplice could set out in the middle of the highway at the last second and because the self-driving car realized two lives are better than one the car veers off the highway.

TylerLinner > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 14:29

1

I think we’re pretty much on the same page here about ethics, now that it’s worded in a serious manner. No, the car should not sacrifice its passengers if a ped is being reckless*. But this is the root problem with autonomous cars which as been discussed ad nauseum for years.

As for crosswalk deserts, the one big one that I was thinking of involves a corporate campus that covers a square mile; there is no good way to cross one of the bordering roads to get lunch, even by car. Bad urban planning, I think.

When the sidewalk ends, it would be nice to have a “dead end” sign. However, that would be a short term solution at best. How about just fixing the damn sidewalk? I wonder if the municipality could ask the community whether they would pay a little extra in taxes to have usable sidewalks- put it to referendum! Maybe the state could strong-arm cities into doing it.

Make developers build sidewalks on their property (if there are none) before they are allowed to sell it or build on it. That seems pretty common sense to me; property owners are already obligated to maintain their land and clear snow. As for the construction sites, just a simple dead end sign there could do pretty well, too. I’ve seen them, but not many.

*quite a grey area right there, isn’t it?

davesaddiction @ opposite-lock.com > swaptastic 10/26/2015 at 14:29

0

So, if a group of schoolchildren mistakenly enter a crosswalk in front of a car bearing down on them at 40 mph, much too close to stop in time and no room to swerve, it should mow down the kids instead of taking some kind of evasive action that could endanger the driver?

There just aren’t absolutes in some cases. There have been instances where human drivers have sacrificed themselves for the good of others. The question is, should software programmers be allowed to make that choice for others?

Justin Hughes > Sampsonite24-Earth's Least Likeliest Hero 10/26/2015 at 14:34

1

Asimov wrote many, many short stories involving robots whose behavior had gone completely wonky, which was then reasoned out to be due to an “edge case” conflict between the Three Laws. So really, many things could go wrong! (Yes I know your comment was sarcastic.)

swaptastic > davesaddiction @ opposite-lock.com 10/26/2015 at 14:39

0

That is a good point. I shouldn’t have made my point so “matter of fact”. There are scenarios that the computer would need to weigh. I would also hope in the new era of self-driving cars that the amount of sensors/data available would never put anyone in the situation to begin with or would be able to mitigate the risk to the point that no one would die.

Smallbear wants a modern Syclone, local Maple Leafs spammer > TylerLinner 10/26/2015 at 14:42

1

*yup. massive.

Smallbear wants a modern Syclone, local Maple Leafs spammer > TylerLinner 10/26/2015 at 14:45

1

Sidewalk referendum—good idea.

Making developers build sidewalks—also good. I really don't get places (unless they are old) without sidewalks. I mean, what's the point?

davesaddiction @ opposite-lock.com > swaptastic 10/26/2015 at 14:47

0

I don’t think it’s possible to take deaths to zero, unless all cars are driving around with no passengers on streets with no pedestrians...

Another wrench in the mix: what if you want to hit someone with your car? What if that person is firing a gun at you? Will the car protect you in that situation?

TylerLinner > Smallbear wants a modern Syclone, local Maple Leafs spammer 10/26/2015 at 14:51

1

I think it comes down to cheapness. No one wants to be the ‘bad guy’ and ‘cost the developers money’, making the city ‘less friendly to business’. But I really feel that if you’re going to sell a property that is the missing link in a sidewalk, you should be strongly encouraged to put one in. Maybe the city/state coule add a tax onto the house sale, to fund the building of a sidewalk there. Build the sidewalk yourself to the inspector’s satisfaction, and you can skip the fee.

swaptastic > davesaddiction @ opposite-lock.com 10/26/2015 at 14:56

0

Another good point. Your right it is going to be difficult going forward. What if the person were drunk and it was a fake gun?

davesaddiction @ opposite-lock.com > swaptastic 10/26/2015 at 15:01

0

It’s completely impossible to account for every circumstance.

What if it were a kid and a fake gun? Or a real one...?

The more you talk about it, the crazier it seems to completely take control away from a human driver (which is what Google wants to do - no steering wheel or pedals at all).